Testing Conventional Wisdom

In my recent article, ‘Do Human Rights Treaties Make a Difference?’, I presented evidence and arguments that called into doubt two widely shared assumptions: (1) that countries generally comply with their human rights treaty commitments and (2) that countries’ practices will be better if they have ratified treaties than they otherwise would be. In response, Professors Ryan Goodman and Derek Jinks argue that we must stick with ‘conventional assumptions’ until we ‘know’ the ‘real effects of human rights treaties’. In this reply, I clarify my argument, which Goodman and Jinks misportray, and respond to the central themes of Goodman and Jinks’ critique. First, I argue that Goodman and Jinks’ scepticism toward my empirical results is misplaced and that their claims that the multiple data sources on which I draw one ‘bod’ are unsubstantiated. Their argument, taken to its logical conclusion, would counsel against any empirical analysis of the effectiveness of human rights treaties. Second, I defend my theoretical account, which argues for looking beyond existing models in analysing state behaviour. Third, I contest Goodman and Jinks’ claim that it promotes human rights to continue to rely uncritically upon conventional assumptions. I argue that the international legal community should instead seek to understand better the relationship between treaties and state behaviour and then carefully consider how to make treaties achieve their goals more effectively. For too long, debates within international human rights law have taken place in the absence of much concrete evidence about the effects of human rights laws on state action. Policy-makers, scholars and activists have all tended to operate on the basis of two unexamined assumptions: that countries generally comply with their treaty commitments and that countries’ practices will be better if they have signed human rights treaties than they otherwise would be. In my recent article, ‘Do Human Rights Treaties Make a Difference?’, I put these assumptions to the test and found evidence suggesting that they are not always correct. Responding to my article, Professors Ryan Goodman and Derek Jinks argue that we must stick with ‘conventional assumptions’ until we ‘know’ the ‘full effects of human rights treaties’. I agree with Mendip Communications Job ID: 9407BK--0183-4 7 186 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 186 EJIL 14 (2003), 185–199 3 Ibid., at 182–183. Goodman and Jinks that treaties remain an indispensable tool for the promotion of human rights. I disagree, however, that it is in the interests of human rights to continue to rely uncritically upon conventional assumptions. In my article, which is but a piece of a broader project, I aimed to begin a conversation about the effectiveness of human rights treaties and how it might be improved. In it, I made four arguments: (1) Empirical analysis can serve as an important tool for understanding the relationship between treaties and state behaviour. (2) Existing theories of international legal compliance are unable to explain the empirical evidence regarding state behaviour. (3) This failure is due, at least in part, to a key oversight. Existing theories fail to see that countries comply (or fail to comply) with treaties not only because they are committed to or benefit from the treaties, but also because they benefit from what ratification says to others — what I term the expressive effect of treaties. (4) Strengthening the monitoring and enforcement of human rights treaties could make it more difficult for countries to use ratification to express a commitment to human rights while making no improvements in their behaviour. Goodman and Jinks respond in three principal ways. First, and most important, they express deep scepticism toward large-scale quantitative empirical analysis, particularly within the field of human rights. This scepticism appears to be fuelled in part by their evident discomfort with the results I reach, which throw conventional assumptions into doubt, and is exacerbated by clear misunderstandings of my argument. Second, they raise objections to my theoretical contribution, which calls for scholars to look beyond traditional bounds in analysing state behaviour. And, third, they question my policy proposal, which calls for a reassessment of the current approach to protecting human rights. Rather than address each point raised by Goodman and Jinks, I seek here instead to clarify the argument of my article and respond to the three central themes of their critique. First, I argue that Goodman and Jinks not only ignore much of the empirical evidence presented in my article, but also utterly fail to demonstrate that the asserted errors in my data are present or that, even if present, would unacceptably bias the results. Second, their complaints regarding my theoretical argument fail to acknowledge that I, too, argue that incorporation of human rights norms is a process in which treaty law can and does play an important role. I differ from them not because I deny this fact, but because I have a different argument about how and why it matters. Finally, their complaints regarding my policy proposals exaggerate our differences. I share Goodman and Jinks’ belief that human rights treaties can have a powerful impact on countries’ human rights practices by changing discourse about rights. What I question is the assumption that weak enforcement and monitoring systems are necessary — or always helpful — to that goal. It is my hope that this exchange will form the beginning of a conversation among scholars, activists and policy-makers that will lead to further improvements in our understanding of the impact of human Mendip Communications Job ID: 9407BK--0184-4 7 187 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 187 4 These misreadings are symptomatic. Throughout their response, Goodman and Jinks misread my claims in ways that make them appear more radical than they are. For example, Goodman and Jinks claim that I argue that ‘international actors (including states and non-governmental organizations) reward ratifying states by reducing political pressure to promote human rights standards, thereby actually increasing human rights violations.’ Ibid., at 172. That is not what I argue. Instead, I claim that ratification can lead international actors to reduce political pressure for real improvements in human rights practices. Hence, my argument does not hinge on a claim that practices worsen in countries that ratify treaties. Rather, it is likely that the results come about from a failure among some such countries to make improvements in their human rights practices that they otherwise would have made. Similarly, at the start of their critique of my theoretical model, Goodman and Jinks claim that I ‘suggest that tests of statistical significance are not as relevant to [my] research design.’ Ibid., at 178, n. 30. In the footnote they cite, see Hathaway, supra note 1, at n. 195, I intended to suggest just the opposite: tests of statistical significance are relevant to my study even though my dataset includes close to the full population of nations because the data can be seen as a ‘sample’ across time. 5 Goodman and Jinks, supra note 2, at 172 (citing Hathaway, supra note 1, at 1999). 6 In the section to which Goodman and Jinks cite, I state: ‘First and foremost, although countries that ratify treaties usually have better ratings than those that do not, noncompliance appears common. Indeed, those with the worst ratings sometimes have higher rates of treaty ratification than those with substantially better ratings. Second and relatedly, treaty ratification is not infrequently associated with worse, rather than better, human rights ratings than would otherwise be expected. Unexpectedly, treaty ratification is more often associated with worse human rights ratings in areas where rights are deeply entrenched in international law than in areas that are of more recent provenance. Third, noncompliance appears less common and less pronounced among countries that have ratified the Optional Protocol to the Covenant on Civil and Political Rights and Article 21 of the Torture Convention, and countries that have ratified these provisions generally have substantially better human rights ratings than those that have not. However, it is possible that this is due largely to a greater proclivity among those with better practices to sign the provisions rather than to the effect of the provisions on state behaviour. Fourth, ratification of regional treaties appears to be more likely than ratification of universal treaties to be associated with high rates of noncompliance and with worse human rights practices than would otherwise be expected. Finally, full democracies appear to be more likely to comply with their human rights treaty obligations than the group of nations as a whole and more likely when they ratify treaties to have better practices than otherwise expected.’ Hathaway, supra note 1, at 1999–2000 (footnotes omitted). rights treaties on countries’ human rights practices and ultimately to a stronger and more effective international human rights regime. 1 Clarifying the Argument It is important to begin by emphasizing that Goodman and Jinks’s response reflects a fundamental misreading of my central arguments. Goodman and Jinks assert that my article makes four empirical claims: ‘(1) countries with worse human rights records appear to ratify treaties at a higher rate than those with better records; (2) treaty ratification is associated with worse human rights practices than expected; (3) enforcement procedures reduce non-compliance; and (4) ratification is associated with better practices in full democracies.’ The best of these descriptions rob my arguments of their nuance and the worst simply get them wrong. First, I do not claim, as they assert, that ‘countries with worse human rights records appear to ratify treaties at a higher rate than those with better records’. Indeed, I claim Mendip Communications Job ID: 9407BK--0185-4 7 188 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 188 EJIL 14 (2003), 185–199 7 Hathaway, supra note 1, at 1976–1988. 8 Ibid., at 1999–2000. 9 Ibid. 10 Ibid. 11 Ibid. just the opposite — on the whole, those that ratify human rights treaties appear to have better practices than those that do not (as revealed in Tables 1 and 2 of the article) and those with better practices are usually (though, as I point out, not always) more likely to ratify human rights treaties than those with worse practices (as revealed in Figures 1–5). I do observe that in some contexts, countries with the poorest practices ratify at rates approaching those of countries with the best practices and that in some instances, countries with marginally worse practices ratify specific treaties at rates slightly higher than those with worse practices. Yet this does not support the blanket assertion that Goodman and Jinks attribute to me. Second, I do not make the unqualified claim that ‘treaty ratification is associated with worse human rights practices than expected’. In the article, I argue not that treaty ratification is associated with worse human rights practices than expected but instead that all other things being equal, ratification of a human rights treaty is sometimes associated with worse measures of human rights practices. The claim, read out of context as it is by Goodman and Jinks, leads to entirely different, and entirely unfounded, implications. Third, I do make what I believe to be the relatively uncontroversial claim (Goodman and Jinks certainly make no effort to disprove it) that stronger monitoring and enforcement procedures have the power to improve compliance with human rights treaties. Specifically, I observe that ‘noncompliance appears less common and less pronounced among countries that have ratified the Optional Protocol to the Covenant on Civil and Political Rights and Article 21 of the Torture Convention, and countries that have ratified these provisions generally have substantially better human rights ratings than those that have not.’ I do caution, however, that ‘it is possible that this is due largely to a greater proclivity among those with better practices to sign the provisions rather than to the effect of the provisions on state behavior.’ Fourth, they miss my empirical observation that ‘ratification of regional treaties appears to be more likely than ratification of universal treaties to be associated with high rates of noncompliance and with worse human rights practices than would otherwise be expected.’ And finally, while their claim that I show that ‘ratification is associated with better practices in full democracies’ is literally correct, their summary creates the incorrect perception that I claim that ratification is not associated with better practices among all countries — a point to which I respond above. It also fails to acknowledge my related claims that ‘full democracies appear to be more likely to comply with their human rights treaty obligations than the group of nations as a whole and more likely when they ratify treaties to have better practices than otherwise expected.’ Mendip Communications Job ID: 9407BK--0186-4 7 189 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 189 12 Ibid., at 1982–1987. 13 Convention on the Prevention and Punishment of the Crime of Genocide, adopted 9 Dec. 1948, S. Exec. Doc. O, 81–1 (1949), 78 UNTS 123 (entered into force 12 Jan. 1951). 14 Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment, opened for signature 10 Dec. 1984, Art. 22, S. Treaty Doc. No. 100–20, at 27–28 (1988), 1465 UNTS 85, 120 (entered into force 26 June 1987). 15 Hathaway, supra note 1, at 1982, 1984. 16 Goodman and Jinks, supra note 2, at 173. 17 Ibid., at 173. 18 Vienna Convention on the Law of Treaties, opened for signature, 23 May 1969, Art. 2, 1155 UNTS 331. 19 Goodman and Jinks, supra note 2, at 172. 2 Defending Empiricism Before turning to Goodman and Jinks’s complaints with the empirical analysis, I must note that their response essentially ignores half of the empirical evidence presented in my article. Even if Goodman and Jinks’s claims regarding the multivariate quantitative analyses were correct (which, as I will shortly demonstrate, is not the case), the findings reported in the first half of the empirical section of the article would still pose a challenge to existing accounts of treaty compliance. The fact remains that in every area that I examine, more than a third of the countries with the worst reported practices had ratified a convention prohibiting those practices. It is stunning that, for example, 47 per cent of countries that are reported to have committed between 64,000 and 128,000 acts of genocide had signed the Convention on the Prevention and Punishment of the Crime of Genocide at the time of the violations and roughly 40 per cent of countries in which torture is reported to be widely practised had signed the Convention against Torture and Other Cruel, Inhuman or Degrading Treatment or Punishment at the time of the violations. Hence, even if treaty ratification does lead to wider reporting of prohibited practices, it remains the case that large numbers of countries with the worst human rights practices are members of treaty regimes that prohibit the practices in which they engage. Existing accounts of treaty compliance remain at a loss to explain these findings. The account I present seeks to fill this gap. Goodman and Jinks not only fail to offer an alternative explanation, they ignore the evidence altogether. Instead, Goodman and Jinks begin their critique of my empirical analysis on a relatively trivial point. They question my choice of ratification as an independent variable in the analysis, noting that ‘ratification is a point in the broader process of incorporation’ and that core treaty obligations actually ‘attach earlier in the incorporation process — that is upon signature of the treaty’. I chose to use ratification (or, where appropriate, acceptance, approval or accession) of the treaty as the independent variable not because I believe it constitutes a ‘magic moment’ of treaty acceptance, but because it is the point at which the state ‘establishes on the international plane its consent to be bound by a treaty’. Hence, treaty ratification is not simply ‘a proxy for the formal acceptance of international human rights law’, it is the formal acceptance of international human rights law. If anything, they ought to applaud the choice of this date over the date of signature, for if states gradually Mendip Communications Job ID: 9407BK--0187-4 7 190 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 190 EJIL 14 (2003), 185–199 20 Goodman and Jinks’ make the related claim that an important event in the ‘incorporation process for a given state may be the decision of another country to ratify a significant human rights treaty’. Goodman and Jinks, supra note 2, at 173. I certainly agree. Unfortunately, contrary to their assertion, Goodman and Jinks, supra note 3, at n. 13, there are no good existing empirical models for examining this behaviour. For this reason, Victor Aguirregabiria and I are currently working to develop a discrete choice model to measure strategic and social interactions in the decisions of states to join human rights treaties. 21 Goodman and Jinks, supra note 2, at 175. 22 Hathaway, supra note 1, at 1939–1940, 1964–1966, 1967–1968, 2000. 23 Ibid., at 2020–2022. 24 Ibid., at 2021. improve their practices from the moment they intend to commit to the treaty, counting membership from a later date would make it more likely that I would detect this positive movement. Goodman and Jinks then present what can only be read as a broad challenge to the use of quantitative analysis to examine the effect of treaties on the practices they are intended to govern. Their claim that ‘the standard variables in this field only measure recorded and reported human rights violations, not actual violations’ is certainly true. But this is of course a critique that can be levelled at any empirical project. It is never possible to measure ‘actual’ practices. By the very nature of the enterprise, there will always be some disjuncture between what has actually occurred and what has been ‘recorded and reported’. This is true not only of large-scale quantitative analysis such as that performed in my article, but of qualitative case studies as well. Because it is not possible ever to know with certainty what ‘actually’ occurred, social scientists that use empirical techniques are always constrained to examining ‘recorded and reported’ practices. In other words, whenever measurement occurs, measurement error will necessarily be present. That is not to say that measurement error is irrelevant. In any empirical project, measurement error is always an issue to which the researcher must be attentive — a point I myself state repeatedly in the article. Short of employing an army of researchers to gather data on the ground (which in this context would not only be infeasible given the worldwide scope of this project but would also mean giving up on using existing historical data), a researcher is left with the data that exist. The question then becomes whether it is possible to use the existing data to produce results that are not unacceptably biased by measurement error. Here, there are several reasons to believe that is possible and hence that measurement error does not have the confounding effect on my findings that Goodman and Jinks hypothesize. First and foremost, Goodman and Jinks’s critique relies not simply on the claim that the existence of human rights treaties increases the salience and legitimacy of human rights concepts — a claim that I myself make in the article. Rather, their argument rests on the narrower and unfounded assumption that this effect is limited to the countries that ratify human rights treaties. While a human rights treaty will undoubtedly ‘influence individual countries’ perceptions of what constitutes acceptable behavior’ and assist non-governmental organizations (NGO) in their struggle to bring attention to human rights abuses, this effect is not limited to ratifying Mendip Communications Job ID: 9407BK--0188-4 7 191 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 191 25 Nothing in Thomas Risse and Kathryn Sikkink’s analysis of this issue, on which Goodman and Jinks rely, Goodman and Jinks, supra note 2, at 176 nn. 19–20, suggests as much. T. Risse and K. Sikkink, ‘The Socialization of International Human Rights Norms into Domestic Practices: Introduction’, in T. Risse et al. (eds), The Power of Human Rights: International Norms and Domestic Change (1999), at 25–28, 238 (discussing ‘Phase 3’ of the process of norms cascades and not once mentioning international human rights treaties or treaty ratification), cited in Goodman and Jinks, supra note 2, at 173 n. 11. To the contrary, Risse and Sikkink emphasize the transnational nature of NGOs. 26 The other treaty-specific processes that Goodman and Jinks point to are equally unavailing. Goodman and Jinks suggest, without citing any supporting evidence, that ‘[i]n many jurisdictions, treaty ratification makes possible the initiation of individual legal claims based on the treaty’s substantive guarantees’. Goodman and Jinks, supra note 2, at 176. I suspect, however, that the number of jurisdictions in which practices are poor and where treaty ratification enables successful individual legal claims based solely on the treaty’s substantive guarantees are small. Goodman and Jinks also claim that the self-reporting requirements attached to UN human rights treaties mean that ratifying countries’ human rights records are more exposed than those of non-ratifying countries. Ibid., at 177. They fail to acknowledge, however, that the self-reporting mechanism to which they point is regularly flaunted without consequence — a shortcoming that I argue should be addressed. Hathaway, supra note 1, at 2023. 27 Hathaway, supra note 1, at 2021. 28 To quote Goodman and Jinks (quoting Robert Justin Goldstein), ‘[I]t is virtually an axiom that the more repressive the regime, the more difficult it makes access to information about its human rights atrocities to researchers.’ Goodman and Jinks, supra note 2, at 175 (quoting Goldstein, ‘The Limitations of Using Quantitative Data in Studying Human Rights Abuses’, in T. B. Jabine and R. P. Claude (eds), Human Rights and Statistics (1992)). 29 Goodman and Jinks, supra note 2, at 175 (quoting Kenneth Bollen’s statement that ‘it is possible that a nation which is relatively open may appear lower in rights and liberties simply because violations are more likely to be reported to the outside world’). 30 Bollen, ‘Political Rights and Political Liberties in Nations: An Evaluation of Human Rights Measures, 1950 to 1984’, in T. B. Jabine and R. P. Claude (eds), Human Rights and Statistics: Getting the Record Straight (1992) 205. countries. Hence, my disagreement with Goodman and Jinks lies in their underestimation of the constitutive effect of human rights treaties. In contrast to their assertion that the effects are limited to ratifying countries, I argue that it extends to ratifying and non-ratifying nations alike. Second, Goodman and Jinks utterly ignore the inclusion in my analysis of a control variable for level of democracy. Because more democratic nations tend to be more open to external and internal scrutiny than less democratic nations (because they enjoy freer operation of the press and less fettered domestic political activity), the democracy measure serves as at least a partial proxy for the very ‘openness’ that Goodman and Jinks claim I fail to consider. Third, Goodman and Jinks’s argument that measurement error explains my results must rely on the assertion that this measurement error infects all the data sources on which I rely. Yet even Kenneth Bollen — whose comment Goodman and Jinks cite as evidence for their claim of systematic measurement error — notes in the very same essay that ‘none of the criticisms of which I am aware have demonstrated a systematic bias in all the ratings.’ By cross-checking my results across multiple data sources, I ensure that bias that may infect one source will not infect my results as a whole. That I Mendip Communications Job ID: 9407BK--0189-4 7 192 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 192 EJIL 14 (2003), 185–199 31 I draw my data on two of the five areas examined — fair trials and torture — from the State Department reports. Contrary to Goodman and Jinks’s claim, Goodman and Jinks, supra note 2, at 178 at n. 28, the torture measure does not include extrajudicial killing independent of torture — rather extrajudicial killing is only included in the measure when it is the result of severe torture. In other words, it is not the killing that is included in the measure but the severe torture that led thereto. The accompanying critique of the measure of genocide, like many of the other specific critiques in the response, is acknowledged and answered in the article itself. See Hathaway, supra note 1, at 1968–1969. 32 Goodman and Jinks, supra note 2, at 179. 33 See Hathaway, supra note 1, at 2000 n. 213. 34 Moreover, my results do not, as Goodman and Jinks claim, ‘run contrary to [my] theoretical prediction of politically motivated under-reporting post ratification’, Goodman and Jinks, supra note 2, at 179, as my results encompass longer time periods, during which this effect may taper off, as well as cross-national comparisons. Their belief that my observation is inconsistent with my results reveals again a fundamental misreading of the results of my cross-national time series data analyses. 35 Ibid., at 1967–1968. 36 Ibid., at 173. Goodman and Jinks’s response poses a Catch-22 to those seeking to engage in quantitative research on human rights: if one uses measures that include actions that fall outside the strict requirements of the treaties, one is guilty of ‘over-inclusiveness’, ibid., at 178 n. 28. If one uses measures that do not include actions that fall outside the strict requirements of the treaties, one is guilty of ignoring ‘substitutability’, ibid., at 174. The truth is that by using both types of measures and finding similar results for both, one can conclude that it is unlikely that either problem has a substantial effect on the results. 37 International Covenant on Civil and Political Rights, adopted 19 Dec. 1966, S. Exec. Doc. E, 95–2, at 23 (1978), 999 UNTS 171 (entered into force 23 Mar. 1976). find similar results across the different measures strongly indicates that my findings are not the result of bias. Goodman and Jinks critique in particular my reliance upon the State Department’s Country Reports on Human Rights, citing my own observation — gleaned from reviewing over two thousand State Department reports — that the reports appear to give newly ratifying countries somewhat lighter scrutiny in the year of ratification and for a short period thereafter as evidence that the reports are ‘biased’. Yet evidence of a bias in a set of data need not undermine all confidence therein. The question instead is whether the bias is a likely source of spurious results. Here, as I note in the article itself, the apparent bias would favour a positive association and disfavour a negative association between ratification and better human rights practices. Hence, if anything, the negative association I sometimes find is possibly even stronger than the results suggest. Moreover, the consistency of the results across all my data sources suggest that any ‘biases’ unique to the State Department data are not responsible for the results I find. Fourth and related, I use both broader and narrower measures of human rights practices, in part to address the possibility raised by Goodman and Jinks that countries will engage in strategic behaviour by substituting one type of human rights violation for another. For example, I use the broad measures of civil liberties and fair trials to assess compliance with the International Covenant on Civil and Political Rights but the narrower measure of torture (double-checked with reference to the very same broader measure to which Goodman and Jinks allude in their discussion of the substitutability of torture and disappearances) to assess compliance with the Mendip Communications Job ID: 9407BK--0190-4 7 193 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 193 38 See Goodman and Jinks, supra note 2, at 175 fn. 15. I use both the Amnesty and State Department-based Purdue Political Terror Scale to verify some of my results. See Hathaway, supra note 1, nn. 170, 178. I do not rely upon these measures more generally because they cover fewer countries and years than does my coding of the State Department reports and because they include measures of actions that are not prohibited by any of the conventions that I study. 39 Goodman and Jinks, supra note 2, at 175. The Inter-American Convention on the Forced Disappearance of Persons did not enter into force until 28 March 1996. 40 Ibid., at 177 (emphasis in original) . 41 Convention against Torture, supra note 14, Art. 21. 42 Optional Protocol to the International Covenant on Civil and Political Rights, adopted 19 Dec. 1966, 999 UNTS 302. 43 Hathaway, supra note 1, at 1999. 44 Goodman and Jinks, supra note 2, at 175. Goodman and Jinks point to no better comprehensive cross-national historical data on which I might have drawn because none yet exists. 45 Indeed, they come close to suggesting this explicitly when they conclude by stating that ‘[p]erhaps the answer is to discard this type of statistical modelling and adopt a softer kind of empiricism, something more sociological than economic.’ Goodman and Jinks, supra note 2, at 183. Convention against Torture. Again, I find similar results across these various types of measures. This provides strong evidence that substitution effects do not explain my results. More generally, however, it is worth noting that were the substitutability problem exactly as hypothesized by Goodman and Jinks — with countries reacting to the prohibition of certain kinds of human rights violations such as torture, political imprisonment and unfair trials (all of which are prohibited by treaties of the Organization of American States and of the United Nations) by substituting worse violations, such as disappearances, that were not prohibited by a treaty until recently — I would have found a positive association between ratification and improved human rights practices in the narrow areas covered by the treaties. Fifth, if the systematic measurement error Goodman and Jinks hypothesize did exist, one would expect to find ratification to be more frequently associated with higher violation ratings, holding all else equal. And if, as Goodman and Jinks claim, increased reporting were ‘more highly associated with particular thematic issues or with regional systems because those treaties are more effective’, 40 one would expect to find the worst results for Article 21 of the Convention against Torture and the Optional Protocol to the International Covenant on Civil and Political Rights, which include some of the most effective treaty mechanisms I study. To the contrary, however, I find that they are associated with some of the best results, and the results for these provisions are consistently better than for the ‘less effective’ treaties to which they are appended. Goodman and Jinks answer that these efforts are not good enough. Given the state of the data ‘in this field’, the dangers imposed by the possibility of systematic measurement error (which is of course only speculated, not proven) are simply too great. Their critique hence amounts to a declaration that a quantitative analysis of these issues ought not be carried out at all. This conclusion lies at the core of much of their critique and it is the fundamental point on which we disagree. Yes, as with any empirical project, there is a possibility of measurement error in the data this project Mendip Communications Job ID: 9407BK--0191-4 7 194 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 194 EJIL 14 (2003), 185–199 46 Hathaway, supra note 1, at 1940 (‘A second obvious drawback of statistical inquiry is that the accuracy of the analysis necessarily depends on the accuracy of the data on which it rests’); ibid., at 2000 and n. 213. 47 Ibid., at 1963–1976. 48 Ibid., at 1940 (‘[T]o the extent that the data on which my study rests are imperfect, there remains a risk that the conclusions I draw are similarly imperfect’); ibid., at 2020 (‘We must not jump to conclusions about the worth of human rights treaties based solely on the quantitative analysis above’). 49 Ibid., at 2022 (‘One obvious step toward improvement would be to enhance the monitoring of human rights treaty commitments, the current weakness of which may make it possible for the expressive and instrumental roles to work at cross-purposes’). 50 Ibid., at 1939–1940. 51 See ibid., at 1939–1940. 52 Goodman and Jinks, supra note 2, at 183. 53 Although Goodman and Jinks tentatively suggest that ‘[p]erhaps the answer is to discard this type of statistical modelling and adopt a softer kind of empiricism’, Goodman and Jinks, supra note 2, at 183, much of their critique of my analyses could be levied at qualitative case studies as well. See supra Section 1. Moreover, qualitative case studies entail their own drawbacks, most notably the difficulty of generalizing from the small number of cases that can be studied using this method to accurately describe the broader forces at work in the interaction between human rights treaties and countries’ practices. Hence, the answer to Goodman and Jinks’ concerns is not to ‘discard’ one form of empiricism in favour of another, but to use the two in tandem. uses. Yes, given what we know about the imperfections in the data, which I meticulously detail, the results derived therefrom ought be viewed with deep caution. Yes, we most certainly need better data on actual human rights practices; indeed, this ought to be a priority of academics and policy-makers who care about human rights. Yes, there are other ways to examine the relationships discussed in the article that would enhance our understanding of the dynamics I discuss. But no, we ought not abandon efforts to understand the effects of treaties on countries’ practices using all tools available, including quantitative analysis. Large-scale empirical analysis such as that employed in the article permits us to detect large-scale trends that may otherwise go unnoticed. By making it possible to control for the effects of various country characteristics on human rights practices, this type of analysis allows us to isolate more effectively the impact of treaty ratification on country practices. And it makes feasible a scope of analysis that is otherwise impossible. That is not to say, of course, that it is without drawbacks or that it is the only approach that can or should be used to examine the relationship between treaties and country practices. The ‘softer kind of empiricism’ — presumably qualitative case studies — that Goodman and Jinks tentatively advocate can operate in tandem with the kind of large-scale quantitative analysis I conduct to give us a fuller picture of the relationship between treaties and country practices. In short, these methods should be viewed not as substitutes but as complements. 3 The Theoretical Argument This leads me to the concerns that Goodman and Jinks raise in connection with the theoretical argument presented in the article. I will begin this part of my reply where they end. Goodman and Jinks conclude their critique by noting that their ‘fundamenMendip Communications Job ID: 9407BK--0192-4 7 195 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 195 54 Goodman and Jinks, supra note 2, at 181. 55 Hathaway, supra note 1, at 2002. 56 Ibid. 57 Ibid., at 2005. 58 Ibid., at 2005 (emphasis omitted). 59 Ibid., at 2006, 2020–2022. 60 Ibid. 61 Ibid., at 2020. 62 Goodman and Jinks, supra note 2, at 178. tal’ complaint with my theoretical argument is that my ‘explanation raises more questions than it answers’. That was precisely my goal. The article is but a piece of a broader project that will seek to take up many of the questions that they raise — questions that could not possibly be fully answered in a single article. This said, I will attempt to answer a few of Goodman and Jinks’s central points in brief. I argue in my article that we cannot fully understand the relationship between human rights treaty ratification and human rights practices unless we understand that treaties operate on more than one level simultaneously. Putting forward what I term a theory of the dual roles of human rights treaties, I propose that treaties have two functions: they create binding law that is intended to have particular effects, and they express the position of the countries that join them. The expressive side of treaties, in turn, has two aspects. Treaties ‘express the position of the community of nations as to what conduct is and is not acceptable’, and they provide nations with an opportunity to convey a message to others through the act of ratification. The act of speaking (via treaty ratification) holds the potential to transform the understandings and practices of not only the state that engages in that speech (by solidifying, or perhaps opening the door to, internalization of those norms) but also the international community (by shaping the shared understanding of acceptable state practice). Because a substantial body of work already exists on this aspect of the expressive function of treaties and because such changes are not readily detected through large-scale empirical analysis, I focus much of my discussion on countries’ use of treaty ratification to express their individual positions on the norms embodied in the treaties. This use of treaties, I argue, helps explain the specific empirical findings of my analysis — why, that is, ratification of treaties may sometimes be associated with worse practices than would otherwise be expected given other country characteristics. In short, I claim that ‘[w]here, as is usually the case in the area of human rights, there is little monitoring or enforcement, combined with strong pressure to comply with norms that are embodied in treaty instruments, treaty ratification can serve to offset, rather than enhance, pressure for real changes in practices’. In their first critique of my theoretical argument, Goodman and Jinks argue that the quantitative analysis I present ‘does not test, nor is it designed to test, the validity of [my] theoretical model’. I designed the multivariate analyses with an empirical question in mind: Do human rights treaties make a difference in countries’ human rights practices? My primary aim was to test the relative strength of the existing theories of international legal compliance. What I found, however, was that many of Mendip Communications Job ID: 9407BK--0193-5 7 196 Rev: 13-03-2003 PAGE: 1 TIME: 14:42 SIZE: 61,00 Area: JNLS O 196 EJIL 14 (2003), 185–199 63 Ibid., at 179. 64 Ibid., at 179. 65 I develop this further in a forthcoming article, in which I hypothesize that the cost of treaty membership to a country is influenced by the degree to which the country’s practices deviate from the standard of conduct required in the treaty (using many of the same measures of compliance employed in the article discussed herein) and the expected likelihood that the country will observe the requirements of the treaty once it becomes a member of the treaty regime. See Hathaway, ‘The Cost of Commitment’, Stanford Law Review (forthcoming 2003); see also O. A. Hathaway, The Puzzle of Human Rights Treaty Formation: When and Why Do Nations Join Human Rights Regimes? (Feb. 2003) (unpublished manuscript). My initial statistical results consistently bear out this hypothesis. 66 Hathaway, supra note 1, at 1977 and 1980. 67 See ibid., at 2013. Again, this is the subject of ongoing research of mine. See Hathaway, supra note 64. 68 See Hathaway, supra note 1, at 2012–2013. 69 See ibid., at 2008–2009. my empirical results were consistent with none of the pre-existing theories. The obvious question posed by these results was, why? The account I present in the article attempts to begin to answer this question. Rather than offer an alternative comprehensive theoretical model of compliance, I instead aim to cast light on the expressive role of treaties — a role that had to that point been almost entirely ignored and which I argue could help account for the paradoxical results of many of my analyses. In doing so, I seek to supplement, not supplant, existing theoretical accounts. Second, Goodman and Jinks suggest that my account assumes that treaty ratification is ‘virtually costless’ and hence relies on the logical impossibility of a ‘costless signal’. International legal commitments only constitute signals, they claim, if they are meaningful commitments; commitments are signals precisely because they are not costless. Contrary to Goodman and Jinks’s suggestion, I do not claim that treaty ratification entails no costs. Rather, I argue that because enforcement of most human rights treaties through military or economic sanctions is rare, the costs of treaty ratification to countries are largely internally imposed. If ratification is not necessarily costly, why do observers take it as a positive indicator of human rights practices? I argue that they do so in part because, as I show in Tables 1 and 2 of the article, countries that ratify human rights treaties do, on average, have better human rights practices than those that do not. Moreover, ratification does carry the possibility of reputational costs. Although the threat of economic or military sanctions is usually small, the threat to a country’s reputation posed by the possibility that it will be revealed to have acted in violation of a treaty to which it has committed may be large — albeit, the degree to which this is true depends centrally on the value the country places on that reputation. The audience may also reluctantly accept what it suspects to be imperfect information regarding countries’ human rights practices because there are relatively few alternative sources of information. And it is possible that some of the audiences for the ratification may treat it as a reliable signal in part because they simply want surface evidence of commitment to the norms embedded in the treaties that they can use to placate more genuinely interested parties to which they must answer. Mendip Communications Job ID: 9407BK--0194-4 7 197 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 197 70 Goodman and Jinks suggest that I misspecify the claims of realist theory, claiming that ‘[f]or realists, rules constrain and facilitate state behaviour without reconfiguring state interests and preferences. It is in this way that international rules have no autonomous and causal status.’ Goodman and Jinks, supra note 2, at 181 n.35. The description Goodman and Jinks offers would be an accurate description of institutionalist theory, but it is certainly not an accurate description of realist theory. Compare Hathaway, supra note 2, at 1944–1947 (describing a realist approach to compliance), with ibid., at 1947–1952 (describing an institutionalist approach to compliance). Again, the source they cite contradicts their claim. As Beth Simmons writes, ‘To realists . . . [i]nternational agreements lack restraining power. . . . In short, realists typically assume that international law is merely an epiphenomenon of interests or is only made effective through the balance of power.’ Simmons, ‘Compliance with International Agreements’, Ann. Rev. Pol. Sci. (1998) 75, 79 (summarizing ‘compliance and the realist tradition’). 71 Hathaway, supra note 1, at 2002–2003 (emphasis added). 72 Ibid., at 2006. Finally, Goodman and Jinks press me to further specify the underlying theoretical presuppositions of my approach. When I argue that realist theory cannot explain the instances in which I find treaty ratification associated with worse practices than otherwise expected given other country characteristics, do I mean to suggest that the underpinnings of the realist approach are flawed? When I claim that the normative approaches overpredict the extent of treaty compliance, am I arguing that their theoretical assumptions are fundamentally flawed? And, finally, if I disconfirm the predictions of existing theories on whose theoretical assumptions I rely, do I not thereby undermine my own argument? The short answer to all three questions is ‘no’. The article suggests that the pre-existing theoretical accounts of countries’ compliance behaviour sometimes result in inaccurate empirical predictions. The obvious implication is that all of these accounts are in some way flawed. But rather than attack any of the underlying theoretical assumptions on which these models are based, I instead suggest that a source of the failure of all the models may be myopia — a failure to see the entirety of the ways in which treaties influence and are influenced by state behaviour. As I put it in the article, none of the pre-existing accounts ‘considers the possibility that countries comply (or fail to comply) with treaties not only because they are committed to or benefit from the treaties, but also because they benefit from what ratification says to others.’ Hence I do not attempt to adjudicate between the many existing accounts of treaty compliance and the theoretical commitments they represent. I do, however, point out a shared flaw and suggest a possible source therefor. But Goodman and Jinks will likely not be satisfied with this answer, for they challenge me to lay bare my own theoretical commitments. Where exactly, they ask, do my own theoretical allegiances lie? At the risk of vastly oversimplifying, let me attempt a brief answer. My work draws on the schools of thought I dub ‘rationalist’ as well as those I consider more ‘normative’. The first expressive aspect of treaties, which arises because treaties express the position of the community of nations as to what conduct is and is not acceptable and thus holds the potential to ‘change discourse about and expectations regarding country practices and thereby change practices of countries regardless of whether they ratify the treaties,’ is more normative in nature. Mendip Communications Job ID: 9407BK--0195-4 7 198 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O 198 EJIL 14 (2003), 185–199 73 See, e.g., Hathaway, ‘Positive Feedback: The Impact of Trade Liberalization on Industry Demands for Protection’, 52 Int’l Org. (1998) 575 (putting forward a dynamic theory of trade preferences). 74 This is, indeed, the central insight of path dependence theory, on which I have written in an entirely different context. See Hathaway, ‘Path Dependence in the Law: The Course and Pattern of Change in a Common Law Legal System’, 86 Iowa L. Rev. (2001) 601 (developing and applying three strands of path dependence theory to explain change in common law legal systems and to offer a positive and normative account of stare decisis). See also, e.g., Koh, ‘Why Do Nations Obey International Law?’, 106 Yale L.J. (1997) 2599, 2655 (arguing that ‘institutions become “carriers of history”, and evolve in pathdependent routes that avoid conflict with the internalized norms’). 75 Hathaway, supra note 1, at 2022. 76 Ibid., at 2023. 77 Ibid., at 2024. The second, which arises from what membership in a treaty regime says about the parties to the treaties and enables countries to use treaties to engage in position-taking that will further their rational self-interested aims, is more rationalist in tone. These two effects work in tandem in ways that I begin to detail in the article and which I will explore further in forthcoming work. Of course, one might argue that these commitments are incompatible. This is, in my view, obviously wrong. It is possible to build a theory in which interests and ideas both matter. Indeed, I would go so far as to say that any theory that does not take both seriously is necessarily incomplete. States and state actors do engage in rational self-interested behaviour. In making decisions, a central consideration of states is the effect that each alternative mode of action will have on their well-being. But states and state actors are also deeply and profoundly influenced by norms and ideas. At a minimum, norms and ideas shape actors’ conceptions of what is in their self-interest and what constitute the bounds of acceptable conduct. Those conceptions are not static but instead change in part through interaction among and between actors. Moreover, ideas are embedded within institutions — including, of course, human rights treaty regimes — which then shape and influence state interests and behaviour. Hence states may be motivated by rational self interest, but their actions are also shaped by norms and ideas. 4 Strengthening the Human Rights Regime I conclude my article with a proposal aimed at strengthening human rights regimes. I suggest that to combat widespread non-compliance with human rights treaties, a first step ‘toward improvement would be to enhance the monitoring of human rights treaty commitments’, and, to that end, I advocate the strengthening of the self-reporting system that currently serves as the backbone of the majority of human rights treaties. I further contend that the findings of my study may ‘give reason to reassess the current policy of the United Nations of promoting universal ratification of the major human rights treaties’ on the grounds that pressure to ratify, if not followed by strong enforcement and monitoring, may be counterproductive. I caution, however, that any reforms aimed at enhancing the effectiveness of treaties must be Mendip Communications Job ID: 9407BK--0196-4 7 199 Rev: 12-03-2003 PAGE: 1 TIME: 14:46 SIZE: 61,00 Area: JNLS O Testing Conventional Wisdom 199 78 Ibid. 79 Goodman and Jinks, supra note 2, at 182. 80 Hathaway, supra note 1, at 2005, 2020–2022. 81 Ibid., at 2024–2025. made with great care, as such changes bear the potential to reduce participation in the treaty regimes. In their final critique, Goodman and Jinks argue that my proposal to enhance the monitoring and enforcement of treaty obligations would undermine the very effectiveness of human rights treaties that I seek to enhance. Universal or broad-based ratification fosters the salience and legitimacy of human rights norms in the international community, they claim. By reducing the ‘opportunities for “shallow” ratification by problem countries, [I] would undermine the considerable constitutive effects of these treaties.’ As I discuss at length in the article, I, like Goodman and Jinks, believe that human rights treaties can and do change perceptions of what constitutes acceptable behaviour and thereby can have a powerful impact on countries’ human rights practices. I therefore share Goodman and Jinks’s concern that any reforms aimed at enhancing the effectiveness of treaties not be made haphazardly. Yet I do not agree that fostering a system of ‘shallow’ ratification is necessary for, or always helpful to, the process of building national human rights cultures. Broad membership in the Optional Protocol to the Covenant on Civil and Political Rights, Articles 21 and 22 to the Torture Convention, and the European Convention on Human Rights belies Goodman and Jinks’ assumption that stronger treaties will necessarily be shunned. Moreover, to the extent that non-compliance with many human rights treaties is widespread and accepted with little formal comment or complaint, the power of those treaties to change discourse and expectations is weakened.